It is important for companies to find cost effective ways to manage their digital file storage. Although it may seem that file storage is inexpensive, 80% or more of the total cost of ownership is in managing and administering that storage. Most organizations' need for file storage is growing at 40% to 50% per year, along with the cost to manage that storage. Today, many companies have so much data that moving it from place to place can be cost-prohibitive.
A number of storage software vendors provide solutions that will store and organize data. Examples of such solutions in include conventional NAS, SAN or DAS storage devices which are typically deployed and maintained by an enterprises IT department. In addition, there is currently a trend towards public and private cloud-based or virtual data stores and associated name spaces supported internally and externally, and accessed by users via a Wide Area Network such as the Internet and by legacy protocols such as CIFS and NFS. Examples of these approaches include the Microsoft® SharePoint™, ByCast, and Xanet services, etc.
One of the drawbacks the Storage Industry has today is that, unlike in the past when file data was comparatively small could be easily copied from one location to another, today's enterprises often have too much data to move other than by necessity. This may be particularly problematic for relatively large users attempting to migrate from conventional user-supported NAS, SAN or DAS storage devices, to the aforementioned cloud-based or virtual data stores. Indeed, for an enterprise-class customer that may have several terabytes (or more) of data, such movement may not be realistically feasible, since the resources required for such a data migration may approach or exceed the available resources of their IT infrastructure.
For example, the US military has recently attempted to standardize on SharePoint™. In total there are approximately 3 million users, hundreds of petabytes of data and trillions of files. Currently, it may be possible to load a trillion records into a database. Indeed, in some applications it may be possible to manipulate a billion records using a conventional desktop computer. However, it is impractical, if not substantially impossible, to move 100 petabytes of data electronically from point A to point B in any reasonable period of time or affordable cost.
Accordingly, what is needed is a cost-effective system and method for the virtual, or post-facto migration of relatively large amounts of data to cloud-based data sharing services or other content management systems.